System and method for selecting electronic documents from a physical document and for displaying said electronic documents over said physical document

ABSTRACT

The present invention discloses a system and a method for selecting, accessing and retrieving directly from a physical (i.e., hard-copy) document, hypertext documents located on a user workstation or on one or a plurality of servers connected to a communication network, and for displaying the retrieved hypertext documents over said physical document. The information is accessed by the user simply by touching with a finger items (i.e., words, letters, symbols, pictures, icons, . . . ) that are highlighted on an electronic hypertext document displayed over the physical document. The method, for use in a user system comprises the steps of: —identifying a physical document, this physical document comprising one or a plurality of pages; —identifying a page of said physical document; —identifying and locating an electronic document associated with the identified page referring to a document hyperlink table; said document hyperlink table comprising for each page of the physical document the identification and location of an electronic document; —accessing and retrieving the electronic document associated with the identified page; —displaying the retrieved electronic document by means of an opto-touch foil placed over the identified page.

RELATED APPLICATION

This application is a national stage application of and claims priorityfrom PCT Application PCT/EP02/11126, filed Sep. 12, 2002, which claimspriority from European Application No. 01480109.6, filed Nov. 13, 2001.These disclosures are hereby incorporated by reference herein in theirentireties. The above-referenced PCT International Application waspublished in the English language and has International Publication No.WO 03/042862 A2.

FIELD OF THE INVENTION

The present invention relates to computer interfaces and to new ways ofenhancing paper-based information with electronic information. Moreparticularly, the present invention relates to a system and a method forusing physical documents as interfaces for navigating on a network andbrowsing hypertext documents (i.e., documents that may containhyperlinks to other documents).

BACKGROUND OF THE INVENTION

During the last years, due mainly to the widespread use of personalcomputers and the universal access of millions of users to the WorldWide Web, “multimedia publishing” has veritably exploded. Due to thewidespread penetration of CD-ROM drives an enormous amount of multimediatitles combining text, images and sounds, are now accessible to ownersof personal computers. Interactive electronic services, video-on-demand,and the World Wide Web are providing access to an increasing offering ofmovies, shopping information, games, multimedia documents, electroniccommerce and many other services. In this evolution, an incredibleamount of hypermedia information is today accessible via the Internet onthe World Wide Web.

Notwithstanding the restless progress in computer technology, electronicrecording, processing and displaying of data, multimedia and the Web,the use of paper has not been reduced. In fact, today, most informationthat people continues to read and consult every day is hard-copied,printed or written information, not electronically stored and displayedinformation.

Even when the public's enthusiasm for new computer-based multimediaservices has been seen by many analysts as a threat to the conventionalforms of hard-copied publishing, particularly book publishing, thereality is that reading a book cannot be compared with readingelectronic media. Even when many electronic systems attempt to replacepaper by providing many advantages such as, for example, a better accessto multimedia services, reading paper remains today preferable for mostpeople, whether they are familiar with computers or not. It is difficultto foresee the replacement in the future, of paper books by electronicbooks (e-books) or the realization of a truly “paper-less” office.

People usually prefer to read and browse through paper catalogs,magazines, newspapers, maps and books by flipping through the pages andglancing at pictures and text. A collection of printed color photographscan be much easily and quickly browsed than a sequence of computerscreens.

On publication entitled “The Last Book”, IBM Systems Journal, Vol 36,No. 3 Vol 36, No. 3-1997, by J. Jacobson, B. Comiskey, C. Turner, J.Albert, and P. Tsao of the MIT Media Laboratory, printed books andcomputer screens are compared in the following terms:

-   -   “A book represents a fundamentally different entity than a        computer screen in that it is a physical embodiment of a large        number of simultaneous high-resolution displays. When we turn        the page, we do not lose the previous page. Through evolution        the brain has developed a highly sophisticated spatial map.        Persons familiar with a manual or textbook can find information        that they are seeking with high specificity, as evidenced by        their ability to remember whether something that was seen only        briefly was on the right side or left side of a page, for        instance. Furthermore their haptic connection with the brain's        spatial map comprises a highly natural and effective interface,        when such information is embodied on actual multiple physical        pages.    -   Another aspect of embodying information on multiple,        simultaneous pages is that of serendipity and comparison. We may        leaf through a large volume of text and graphics, inserting a        finger bookmark into those areas of greatest interest.        Similarly, we may assemble a large body of similar matter in        order to view elements in contrast to one another, such as might        be done to determine which of a particular set of graphical        designs is most satisfying”.

For people, paper has many advantages; paper is portable, familiar andeasily distributed; paper can be easily viewed, marked, or manipulated,and does not require energy source to display. The friendliness,usefulness and availability of hard-copy documents cannot be reproducedtoday with electronic documents.

However, the most important problem, of course, with printed books isthat, traditionally, they cannot easily be changed, amended, updated norcompleted.

To make additional information directly accessible from printedpublications, U.S. patent application Ser. No. 09/841930 (IBM'sreference FR9-2000-0022), entitled “Method and system for accessinginteractive multimedia information or services by touching highlighteditems on physical documents” discloses a system and method for selectingand accessing multimedia information and/or services located on a userworkstation (or on one or a plurality of servers connected to acommunication network) simply by touching with a finger, items (words,letters, symbols, pictures, icons, . . . ) that are electronicallyilluminated over the surface of a hard-copy document (or any otherphysical surface) by means of an opto-touch foil. The referenced systemincludes:

-   -   an opto-touch foil preferably transparent, placed by the user        over (or under) the document (or a portion of said document).        This opto-touch foil is used:        -   to illuminate and highlight hyperlinked items over the            surface of the physical document (or portion of this            document), and        -   to read coordinates of these hyperlinked items,    -   an user workstation for accessing and displaying the information        and/or the service associated with the hyperlinked items.

The hyperlinked items are identified by means of a luminous signal (orlight spot) generated by the opto-touch foil. The opto-touch foiloperates under the control of the user workstation. Illuminated itemsare selected by pressing the opto-touch foil. When the user selects anitem among all illuminated items, the user workstation receives from theopto-touch foil a signal indicating the position of this selected item.The user workstation identifies and locates referring to a hyperlinktable the information and/or the service associated with the position ofthe selected item. If the information and/or service is located in aremote server, a request is sent to this server. If the informationand/or the service is stored in the user workstation, then thisinformation and/or service is accessed locally.

The information and/or service is then retrieved and displayed on aseparate display, but not over the document to which this information isrelated. Thus, even if the information associated with illuminated itemsis selected from the physical document, this information is shown to theuser on a separate media, generally on the user's workstation display,but not over the same document. Thus, the user is forced to frequentlyto move his head and to change eyes focus, going forward and backwardsfrom the physical document to the workstation display and vice versa.This process is detrimental for the best user's comfort, rest, andattention.

Using the same opto-touch foil, U.S. patent application Ser. No.09/892399 (IBM's reference FR9-2000-0027), entitled “System and methodfor locating on a physical document items referenced in a electronicdocument”, discloses a method and system for using the same opto-touchfoil for locating on a physical document items referenced in anelectronic document. In a preferred embodiment, this invention enablesto highlight on a paper map the geographic position of places referencedin a Web page.

Also, using opto-touch foils, U.S. patent application Ser. No. 09/923150(IBM's reference FR9-2000-0039), entitled: “System and method forlocating on a physical document items referenced in another physicaldocument”, discloses a method and system for creating hyperlinks fromitems (e.g. words, pictures, foot notes, symbols, icons) on a firstphysical document to particular points on a second physical document(manuscript or printed document), for activating these hyperlinks simplyby touching the first document, and for highlighting, by means of alight emitting source, the position of the items on the second document.In a preferred embodiment, the invention enables to highlight on ahard-copy map the geographic positions of places referenced in ahard-copy document.

These patent applications are limited to show the positions of itemsreferenced on electronic or physical documents, as being simple lightspots over paper maps.

The herein above cited reference, “The Last Book”, IBM Systems Journal,Jacobson et al., describes the possibility of what might be consideredan “animated” book:

-   -   “Finally, a remark should be made about the changing entity of        the book itself. A medieval religious book, for instance, is        immediately identified from the thick, black, Gothic lettering        invented during the time of Charlemagne. Similarly, the richly        drawn first letter of the Beatus page or the poetic layout of        almost any book typeset by the inventor of the modem portable        book, Aldus Manutius, is easily identified . . . . Thus the book        on Arabian horse genetics may have video clips showing the        performance of certain classes of horses. The key is that the        video clip resides, spatially mapped, to a particular page in a        particular book sitting on our shelf. It has a particular        spatial place where we know we can find it. Contrast this with        the single monitor we now have on the desktop, through which all        changeable images must come, and the idea of the animated        manuscript is clear”.

This is a quote from Dr. James Sheats, in the article entitled:“Introduction to Organic Light-Emitting Diodes (OLEDs)”, on therolltronics.com Internet site:

-   -   “The effects of the electronics and photonics revolutions,        enabled by the silicon-based transistor (and its incorporation        into integrated circuits), fiber optics, and solid state lasers,        are evident in almost every aspect of modem commerce. Yet, far        from saturating the market, these devices are predicted to        proliferate far beyond anything we have imagined so far. Whereas        roughly 50% of households in the developed countries own one        personal computer, industry leaders today predict that in the        next decade or so we will all own dozens or even hundreds of        computers (most of them embedded in information appliances), all        of which will communicate with each other on a network similar        to the Internet. We will not know where these computers are, nor        will we care, as long as they carry out their functions.    -   In this new world of “pervasive computing” (a term coined by        Joel Birnbaum of Hewlett-Packard Labs), in which most computing        is carried out by distributed resources connected by a        utility-like network, the user's awareness of a “computer” lies        only in what he or she sees at the interface: the display and        input devices. Displays, now considered a “peripheral,” will be        the central object from the user's perspective, while the        processor becomes peripheral. This vision, however, requires        displays that are far different from the current cathode ray        tubes and expensive (and slow) liquid crystals, since they must        be numerous, compact, and portable. Today display technology is        primitive compared to computing technology. Indeed, paper is the        preferred medium, resulting in the opposite of the early vision        of the “paperless office.”

Therefore, today there is a need to provide users with new systems andmethods for improving physical documents with information in the form ofelectronic graphics, images and/or text that could be displayed incontext, over said physical documents.

The present invention discloses how new display technologies, such asthose based on passive matrix, transparent, organic electroluminescence(EL) devices (TOLEDS) and EL polymers, that are broadly available today,can be integrated with other technologies, like touch-foil technologies,and can be used to transform fixed, static, hard-copy documents intodynamic, animated, changeable, sources of information.

OBJECTS OF THE INVENTION

It is a broad object of the present invention to expand and complementthe information content of physical documents by electronic means.

It is another object of the present invention to select and accesselectronic documents directly from physical documents and to displaysaid accessed electronic documents over the same physical documents.

It is another object of the present invention to use physical documentsas interfaces for navigating in a network and browsing hypertextdocuments.

SUMMARY OF THE INVENTION

The present invention discloses a system and a method for selecting,accessing and retrieving directly from a physical (i.e., hard-copy)document, hypertext documents located on a user workstation or on one ora plurality of servers connected to a communication network, and fordisplaying the retrieved hypertext documents over said physicaldocument. The information is accessed by the user simply by touchingwith a finger items (i.e., words, letters, symbols, pictures, icons, . .. ) that are highlighted on an electronic hypertext document displayedover the physical document.

As defined in independent claims, the present invention is directed to asystem and method for selecting electronic information from a physicaldocument, and for displaying said selected electronic information overthis physical document. This method, for use in a user system comprisesthe steps of:

-   -   identifying a physical document, this physical document        comprising one or a plurality of pages;    -   identifying a page of said physical document;    -   identifying and locating an electronic document associated with        the identified page referring to a document hyperlink table;        said document hyperlink table comprising for each page of the        physical document the identification and location of an        electronic document;    -   accessing and retrieving the electronic document associated with        the identified page;    -   displaying the retrieved electronic document by means of an        opto-touch foil placed over the identified page.

Further embodiments of the invention are provided in the appendeddependent claims.

The foregoing, together with other objects, features, and advantages ofthis invention can be better appreciated with reference to the followingspecification, claims and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel and inventive features believed characteristics of theinvention are set forth in the appended claims. The invention itself,however, as well as a preferred mode of use, further objects andadvantages thereof, will best be understood by reference to thefollowing detailed description of an illustrative detailed embodimentwhen read in conjunction with the accompanying drawings, wherein:

FIG. 1 shows the main components of the invention.

FIG. 2 shows the internal structure of an opto-touch foil.

FIG. 3 shows the steps of the method for hyperlinking a physicaldocument to hypertext documents displayed over said physical document.

FIG. 4 shows the steps of the method for selecting, accessing anddisplaying hypertext documents on physical documents.

FIG. 5 shows a physical (i.e., a hard-copy) document.

FIG. 6 shows a page of a physical document.

FIG. 7 shows how an opto-touch foil is placed aligned over a page of aphysical document.

FIG. 8 shows how a user enters a document number and pages numbers bymeans of sensitive touch buttons located on the opto-touch foil.

FIG. 9 shows how information related to a selected page of physicaldocument is retrieved and displayed over the selected page by means ofthe opto-touch foil.

FIG. 10 shows how a user activates an hyperlink by pressing theopto-touch foil over a selected illuminated item.

FIG. 11 shows how the requested information is received and displayed bythe opto-touch foil over the page of the physical document.

FIG. 12 shows how a user activates another hyperlink by pressing theopto-touch foil over a new selected illuminated item.

FIG. 13 shows how the new requested information is received anddisplayed by the opto-touch foil over the page of the physical document.

PREFERRED EMBODIMENT OF THE INVENTION

The present invention discloses a system and a method for selecting,accessing and retrieving directly from a physical (i.e., from hard-copy)document, electronic hypertext documents located on a user workstationor on one or a plurality of servers connected to a communicationnetwork, and for displaying the retrieved electronic hypertext documentsover this physical document. To select and to get access to additionalinformation, the user simply touches with a finger, items (i.e.,underlines, words, letters, symbols, pictures, icons, . . . ) that arehighlighted on an electronic hypertext document displayed over thephysical document.

The system includes:

-   -   a transparent opto-touch foil, placed by the user over the        physical document (or over a portion of said physical document).        This opto-touch foil is used:        -   to display hypertext documents over the surface of the            physical document (or over a portion of the physical            document);        -   to highlight hyperlinked items comprised in displayed            hypertext documents;        -   to sense the coordinates of hyperlinked items selected by            the user;    -   a user workstation for accessing hypertext documents associated        with selected hyperlinked items and for sending received        hypertext documents to the opto-touch foil for visualization.

The opto-touch foil operates under the control of the user workstation.An hypertext document comprising hyperlinked items is displayed over aphysical document by means of the opto-touch foil, Hyperlinked items arehighlighted to be easily identified. The user selects an highlighteditem simply by pressing the opto-touch foil. When an item is selectedamong all highlighted items, the user workstation receives from theopto-touch foil a signal indicating the position of this selected item.The user workstation identifies and locates the address of the hypertextdocument associated with the position of the selected item.

-   -   If the hypertext document is located in a remote server, a        request is sent to this server.    -   If the hypertext document is stored in the user workstation,        then the hypertext document is accessed locally.

When the selected hypertext document is received, the user workstationores it and controls its display by the opto-touch foil.

System for Selecting, Accessing and Displaying Electronic Documents overPhysical Documents

As shown in FIG. 1, the system according to the present inventioncomprises:

-   -   (101) A physical (hard-copy) document (e.g., a book) comprising        one or a plurality of pages.    -   (102) An opto-touch foil connected to a user workstation. The        opto-touch foil is placed over a page (or a portion) of the        document.    -   (103) A user workstation (preferably connected to a        communications network) for accessing hypertext documents        associated with selected hyperlinked items and for sending        received hypertext documents to the opto-touch foil for        visualization.        Physical Document

Physical documents (101) can be of any kind. They can be, for instance,geographic maps (e.g., topographical maps, political maps, historicalmaps, route maps, shaded relief maps, city maps, natural resources maps,rail road maps or even any other type of map), novel books, text books,technical plans, commercial catalogs or even any other type ofhard-copy, engraved, written, or printed surfaces (e.g., paintings in amuseum of art). The material of the physical documents can be paper,plastic, wood or any other material.

Opto-Touch Foil

In a preferred embodiment, the opto-touch foil (102) comprises two,functionally independent, transparent foils, namely:

-   -   a touch foil, and    -   a light emitting foil (opto foil).

FIG. 2 shows the cross section of an opto-touch foil (200) comprising:

-   -   a transparent resistive or capacitive touch foil (201), of the        type used commonly to manufacture touch screens. The generated        signal is generally proportional to the coordinates (X, Y) of        the point that is pressed or touched (203),    -   a transparent light emitting foil (202), which is a transparent,        bright, self-emitting display that can emit light (204) from        either one or both surfaces.

The combination of both foils (i.e., the touch foil stacked over thelight emitting foil) forms an opto-touch foil (200). FIG. 2 representsan opto-touch foil placed and aligned over a physical document (205)comprising a plurality of items (206) (i.e., words, pictures, icons,etc.) printed (or written, painted, engraved . . . ) on its surface.

The opto-touch foil may communicate (106) with the user workstation(103) over an infrared link, a serial wired connection or any othercommunication means (e.g. by means of a wireless connection operating inthe globally available 2.4 Ghz band of the “Bluetooth” specification, aspromoted by the “Bluetooth Special Interest Group” and documented on theOfficial Bluetooth Website.

Transparent Touch Foil Technology

According to one aspect of the present invention, the touch foilcomponent (201) may be made of transparent resistive or capacitive filmsof the type used commonly to manufacture touch screens. The generatedsignal is generally proportional to the coordinates of the pointpressed. One example of touch foil technology it would be possible touse, is the TouchTek4 (4-wire Analog Resistive Touchscreens) of MICROTOUCH company (TouchTek™ is a trademark of the MICRO TOUCH Company). TheTouchTeck4 technology is fully described in the Web site of MICRO TOUCHCompany.

TouchTek4 touchscreens are commonly used today, in handheld personalinformation management systems, PDAs, mobile computing systems,automotive, diagnostics and telecom devices, and Internet appliances.TouchTek4 touchscreens are engineered to accept more than three milliontouches on any area of the screen.

Transparent Light-Emitting Foil Technology

According to another aspect of the present invention, the transparentlight-emmiting foil (202) may commonly include a substrate having anarray comprising a plurality of transparent scanning lines, transparentdata lines crossing said scanning lines, and electro-luminiscent (EL)elements (pixels) on the intersections of the scanning and data lines.With today's technology, this passive-matrix, light-emitting display maybe made of an array of TOLED's (Transparent Organic Light EmittingDevices) of the types used to create vision area displays onwindshields, cockpits, helmets and eyeglasses.

In its most basic form, a TOLED is a monolithic, solid-state deviceconsisting of a series of “small molecule” organic thin films sandwichedbetween two transparent, conductive layers. When a voltage is appliedacross the device, it emits light. This light emission is based upon aluminescence phenomenon wherein electrons and holes are injected andmigrate from the contacts toward the organic heterojunction under theapplied electric field. When these carriers meet, they form excitons(electron-hole pairs) that recombine radiatively to emit light. As aresult, TOLEDs are bright, self-emitting displays that can be directedto emit from either or both surfaces. This is possible because, inaddition to having transparent contacts, the organic materials are alsotransparent over their own emission spectrum and throughout most of thevisible spectrum.

TOLED displays are today manufactured with standard siliconsemiconductors. Since TOLEDs are thin-film, solid-state devices, theyare very thin, lightweight and durable, ideal for portable applications,like the one disclosed in this invention. TOLEDs can be bottom, top, orboth bottom and top emitting. Also, TOLEDs technology has attractiveadvantages regarding, transparency (TOLED displays can be nearly asclear as the glass or substrate they're on and when built between glassplates, TOLEDs are >85% transparent when turned off), energy efficiency(for longer battery life), full viewing angle, bright and high contrastlight emission, fast response time, and environmental robustness. Thus,TOLEDs are well suited for manufacturing the light-emitting,electro-luminiscent, display component, used jointly with thetransparent touch foil (201) according to the present of this invention.

One example of light emitting foil technology it would be possible touse is the technology used for the TOLEDs manufactured by UNIVERSALDISPLAY Corporation, fully described in the Web site of UNIVERSALDISPLAY Corporation.

User Workstation

The user workstation (103) is used to control the selection, access andretrieval of hypertext documents, stored locally or located on one or aplurality of servers connected to the network, and to control thedisplay of received electronic documents by the opto-touch foil. Theuser workstation include means for requesting, receiving and storinghypertext documents, for receiving the coordinates of the points pressedby the user on the opto-touch foil (102), for identifying the hyperlinkaddresses associated with the points pressed, and for controlling thedisplay by the opto-touch foil (102) of the received hypertextdocuments.

The user workstation can be a Personal Computer, an Internet enabledcell phone, a PDA (Personal Digital Assistant), an onboard computer, anetwork computer, an Internet appliance or a wireless IP enabled device,connected to the opto-touch foil (102). The opto-touch foil (102) maycommunicate with the user workstation by means of a cable (106), a wirepair, an infrared link, or a wireless radio link.

In a particular embodiment, the user workstation is connected to theInternet network (104) and comprises a Web Browser application. Serversare Web servers (105) and the hypertext documents are HTML Web pages.Said HTML Web pages are displayed by the opto-touch foil (102) over thephysical document (101). They are selected by pressing the opto-touchfoil (102) on highlighted items.

Method for Creating Hyperlinks from a Physical Document to HypertextDocuments

As shown in FIG. 3, the method for creating hyperlinks from a handwritten or printed document (like the document shown in FIG. 5 entitled“Interactive Maps of World Cities”), to a plurality of servers to accesselectronic documents to be displayed over this physical document,comprises the steps of:

-   -   (301) assigning a reference number (document identifier) to the        physical document;        for each page of the physical document from which the user        desires to create hyperlinks, to access and display hypertext        documents:    -   (302) selecting a page of the physical document;    -   (303) defining the address of an hypertext document. This        hypertext document will be automatically accessed and displayed        by the opto-touch foil when the user will select this page;        and, for each hypertext document, that the user desires to        display over this page:    -   (304) editing and formatting the hypertext document so that        information can be displayed, with the layout required, over the        page of the physical document, by the opto-touch foil;    -   (305) loading the hypertext document on the corresponding        server.        (301) Assigning a Reference Number to the Document

As shown in FIG. 5, for each document (501) he receives, the user:

-   -   assigns a reference number (502) to this document (e.g., 387)        for identifying said document.    -   writes this reference number (502) on the document, and    -   creates a Document Hyperlink Table, associated with the document        and accessible from the user workstation, said Document        Hyperlink Table comprising the reference number of the document        and other relevant information related to the document such as        title, author, ISBN (International Standard Book Number), URL of        publisher's Web page, date, etc . . . The hyperlink table may be        stored within the user workstation or may be stored in an        external memory accessible from the user workstation.

The header of the new created Document Hyperlink Table associated withthe document shown in FIG. 5, can be built as follows:

Doc: 0387 Title: “Interactive Author: David Nash Maps of Ford, EditorWorld Cities” Binfield, Berkshire, UK URL: http://interactivecities.comDate: 28 Jan. ISBN: 2001 96-375-0556-2 Page: Link: URL:

For each page of the physical document from which it is desired tocreate hyperlinks, for accessing and displaying hypertext documents:

(302) Selecting a Page, and (303) Defining the Address of an HypertextDocument

As shown in FIG. 6, each time the user wants to create one or aplurality of hyperlinks from a page (602) of the physical document (601)to one or a plurality of electronic hypertext documents, he:

-   -   selects a page of the document;    -   enters in the Document Hyperlink Table, means for identifying        said page (preferably a page number, e.g., 25)(603);    -   associates an hypertext document with this page. The user        selects the destination address (e.g.,        http://www.worldcities.com/Paris.html) of the hypertext        document. This destination address identifies, within the        communication network, a server and, within this server, the        hypertext document that must be automatically accessed and        displayed over the page, when this page is selected by the user;    -   enters this destination address (an URL address for example) in        the document Hyperlink Table.

In a preferred embodiment, using the herein described method forhyperlinking the different pages (602) of a same document (601), theDocument Hyperlink Table would appear like this:

Doc: 0387 Title: “Interactive Author: David Nash Maps of Ford, EditorBinfield, World Cities” Berkshire, UK Publisher/ Date: 28 Jan. ISBN:96-375-0556-2 Series URL: 2001 http:// interactivecities.com Page: Link:URL:  1 “Map of http://www.worldcities.com/ Amsterdam” Amsterdam.html  2“Map of Berlin” http://www.worldcities.com/ Berlin.html  . . .  . . .  .. .  24 “Map of Padua” http://www.worldcities.com/ Padua.html  25 “Mapof Paris” http://www.worldcities.com/ Paris.html  26 “Map of Praga”http://www.worldcities.com/ Praga.html  . .  . .  . . 173 “Map ofWichita” http://www.worldcities.com/ Wichita.html

The Document Hyperlink Table may be stored locally in the userworkstation (103) or well it may be remotely stored on a Web server(105) accessible through the Internet network (104) from the userworkstation.

(304) Editing and Formatting Hypertext Documents Accessible from thePage

As shown by FIG. 9, the information (903), (904) comprised in thehypertext documents that can be accessed from a particular page (902) ofa physical document (901), must conform with the layout required so thatthis information when displayed by the opto-touch foil (903)(904),appears to the user correctly aligned and distributed over the physicalpage (902).

To that aim, the hypertext documents must be edited and formattedbeforehand over the layout of the physical page. In practice this can bedone, for example, by editing these hypertext documents over a digitizedcopy (e.g., over a full size scanned copy) of the physical page (902).

(305) Loading Hypertext Documents on the Corresponding Server

Once edited and formatted, hypertext documents related to (i.e.,hyperlinked) a physical page (602) are loaded on servers (105) atcorresponding hyperlink addresses. The hypertext documents are accessedon this servers and retrieved by the user workstation (103) before beingdisplayed over the physical page by the opto-touch foil (102).

Method for Selecting, Accessing and Displaying Hypertext Documents overa Physical Document

The user receives a hand written or printed document, like the documententitled “Interactive Maps of World Cities”, (Doc: 387) shown in FIG. 5.Hyperlinks have been previously defined from selected pages of thisphysical document (101) to hypertext documents located on servers (105)connected to a communication network (104).

As shown in FIG. 4, the method for selecting, accessing and retrievinghypertext documents from a page of a physical document, and forvisualizing these hypertext documents over this physical page, comprisesthe steps of:

-   -   (401) selecting a physical document;    -   (402) selecting a page of this physical document;    -   (403) placing and aligning the opto-touch foil over the selected        page;    -   (404) identifying in the Document Hyperlink Table the address of        the hypertext document associated with the selected page;    -   (405) accessing and retrieving this hypertext document;    -   (406) displaying by means of the opto-touch foil, the retrieved        hypertext document over the selected page of the physical        document;    -   (407) building a Page Hyperlink Table from this hypertext        document;    -   (408) selecting an hyperlinked item by pressing the opto-touch        foil over this item;    -   (409) determining the position on the opto-touch foil, of the        selected hyperlinked item;    -   (410) identifying the address of the selected hypertext document        in the Page Hyperlink Table.        (401) Selecting a Physical Document

By means of any user interface (keyboard, mouse, touch screen, . . . )or any reading means (bar code reader . . . ), the user enters thereference number (identifier) of the document (e.g.: document 387) hewants to select. In the particular embodiment shown in FIG. 8, the userenters the reference number of the document (e.g.: Doc: 387) by means ofan array of pressure sensible “touch buttons” (807) printed on thebottom of the opto-touch foil. The user presses (804) in the followingorder:

-   -   a touch button marked “Doc, and then    -   numerical touch buttons corresponding to each digit of the        document reference number (e.g., “3”, “8” and “7”).

This operation gives access to the Document Hyperlink Table associatedwith the selected document. In the case of the exemplary document shownin FIG. 5, the Document Hyperlink Table would be like this:

Doc: 0387 Title: “Interactive Author: David Nash Maps of Ford, EditorBinfield, World Cities” Berkshire, UK Publisher/ Date: 28 Jan. 2001ISBN: 96-375-0556-2 Series URL: http:// interactivecities.com Page:Link: URL:  1 “Map of http://www.worldcities.com/ Amsterdam”Amsterdam.html  2 “Map of Berlin” http://www.worldcities.com/Berlin.html  . . .  . . .  . . .  24 “Map of Padua”http://www.worldcities.com/ Padua.html  25 “Map of Paris”http://www.worldcities.com/ Paris.html  26 “Map of Praga”http://www.worldcities.com/ Praga.html  . .  . .  . . 173 “Map ofWichita” http://www.worldcities.com/ Wichita.html(402) Selecting a Page of this Physical Document

The user goes to a page of the document (e.g.: page shown in FIG. 6),and, by means of any user interface (keyboard, mouse, touch screen . . .) or any reading means (bar code reader . . . ), enters the page number(603) of the document he wants to select. In the particular embodimentshown in FIG. 8, by means of the array of pressure sensible touchbuttons (807) printed on the bottom of the opto-touch foil (806), theuser enters the page number to select (e.g.: Pg. 25). The opto-touchfoil can be on any position (normally the opto-touch foil is placed overthe selected page). The user presses (805) in the following order:

-   -   a touch button marked as “Pg”, and then    -   numerical touch buttons corresponding to each digit of the page        number (e.g., “2” and “5” ) to select.

This procedure gives access to the entry (shown in reverse video)corresponding to the selected page (e.g., Pg. 25) within the DocumentHyperlink Table associated with the selected document (e.g., Doc: 387).

Doc: 0387 Title: “Interactive Author: David Nash Maps of Ford, EditorBinfield, World Cities” Berkshire, UK Publisher/ Date: 28 Jan. 2001ISBN: 96-375-0556-2 Series URL: http:// interactivecities.com Page:Link: URL:  1 “Map of http://www.worldcities.com/ Amsterdam”Amsterdam.html  2 “Map of Berlin” http://www.worldcities.com/Berlin.html  . . .  . . .  . . .  24 “Map of Padua”http://www.worldcities.com/ Padua.html  25 “Map of Paris”http://www.worldcities.com/ Paris.html  26 “Map of Praga”http://www.worldcities.com/ Praga.html  . .  . .  . . 173 “Map ofWichita” http://www.worldcities.com/ Wichita.html

The Document Hyperlink Table is either stored locally in the userworkstation, or is retrieved from a remote server and then storedlocally in the user workstation or is stored in a remote server and isaccessed remotely.

(403) Placing and Aligning the Opto-Touch Foil Over the Selected Page

As shown in FIG. 7, once a physical document (701) and a page (702) havebeen selected, the opto-touch foil (703) is:

-   -   placed over the page, and    -   aligned (704) with the borders of the selected page by some        conventional means (e.g., by adjusting the lower left corner of        the touch foil with the lower left corner of the page).        (404) Identifying in the Document Hyperlink Table the Address of        the Hypertext Document Associated with the Selected

The Document Hyperlink Table comprises for each hyperlinked page of thephysical document, the location within the network of the hypertextdocument that must be automatically accessed when the page is selectedby the user. The hyperlinked hypertext document may be identified bymeans of a destination address. In the Internet network, Web pages inWeb Servers are identified by an URL (Uniform resource Locator).

In our example, the Document Hyperlink Table associates the “Page 25”,with the hyperlink named: “Map of Paris” located at the URL:http://www.worldcities.com/Paris.html.

(405) Accessing and Retrieving this Hypertext Document

The user workstation (103) activates the hyperlink (destination address,URL, . . . ) of the selected hypertext document.

In our example, the hyperlink labeled “Map of Paris” is activated, andthe hypertext document at the URL: http://www.worldcities.com/Paris.htmlis accessed through the network and retrieved by the user workstation.

(406) Displaying the Hypertext Document Over the Page of PhysicalDocument

As shown by FIG. 9, the information (903) (904) comprised in thereceived hypertext document is displayed over the page (902) of thephysical document (901), under the control of the user workstation.

In the particular embodiment of the invention shown in FIG. 9, items, inthe displayed hypertext document, that are merely informative (903) areshown using normal characters, while items, that are hyperlinked (904)to additional information, are shown underlined. In an alternateembodiment, hyperlinked items can be displayed by the opto-touch foil,for instance, in reverse video.

(407) Building a Page Hyperlink Table from this Hypertext Document

As illustrated in FIG. 9, the received hypertext document can alsocontain hyperlinks (904) to other hypertext documents. To select and getaccess from this physical page (902) to these hyperlinked documents, aPage Hyperlink Table is built on the user workstation, from informationcomprised in the received hypertext document. Basically, the informationof hyperlinks that are encoded on the received hypertext document, isfirst copied into this table. Then, the opto-touch foil coordinates(X,Y) (906) of the hyperlinked items (as they are displayed by theopto-touch oil), are computed and are copied into the correspondingentries of the Page Hyperlink Table.

In the example shown in FIG. 9, the Page Hyperlink Table correspondingto the displayed hypertext document would be like this:

Doc: 0387 Title: “Interactive Author: David Nash PAGE: 25 Maps of Ford,Editor Binfield, World Cities” Berkshire, UK (X, Y): Link: URL: 5 150“Cafeterias” http://www.worldcities.com/ Paris/Cafeterias.html 5 140“Cinemas” http://www.worldcities.com/ Paris/Cinemas.html 5 130“Churches” http://www.worldcities.com/ Paris/Churches.html 5 120“Dancing” http://www.worldcities.com/ Paris/Dancing.html 5 110“Embassies” http://www.worldcities.com/ Paris/Embassies.html 5 100“Hospitals” http://www.worldcities.com/ Paris/Hospitals.html 5 90“Hotels” http://www.worldcities.com/ Paris/Hotels.html 5 80 “Metro”http://www.worldcities.com/ Paris/Metro.html 5 70 “Monuments”http://www.worldcities.com/ Paris/Monuments.html 5 60 “Museums”http://www.worldcities.com/ Paris/Museums.html 5 50 “Pharmacies”http://www.worldcities.com/ Paris/Pharmacies.html 5 40 “Police”http://www.worldcities.com/ Paris/Police.html 5 30 “Pubs”http://www.worldcities.com/ Paris/Pubs.html 5 20 “Post Offices”http://www.worldcities.com/ Paris/Post-Offices.html 5 10 “Restaurants”http://www.worldcities.com/ Paris/Restaurants.html(408) Selecting an Hyperlinked Item by Pressing the Opto-Touch Foil Overthis Item

As shown in FIG. 10, the user selects an hyperlinked item (1003)(“Museums”) on the page (1002) by pressing (e.g., by means of hisfingertip) the opto-touch (1004) foil on this underlined illuminatedword.

(409) Determining the Position of the Point Pressed on the Opto-TouchFoil

The opto-touch foil (1004) sends a signal to the user workstation (103)to identify the selected item. This signal indicates the position on thepage (1002) of the point that has been pressed by the user on theopto-touch foil. The generated signal is generally proportional to thecoordinates (X,Y) of the point that has been pressed.

In our example, the opto-touch foil measures the position on the page ofthe point that has been pressed (near the underlined “Museums”) by theuser. The coordinates that are measured at this point are around X=5 andY=60.

(410) Identifying the Address of the Selected Hypertext Document in thePage Hyperlink Table

The hyperlinked item selected on the opto-touch foil (1004) by the useris identified thanks to the Page Hyperlink Table. This table comprisesan indication of the position (coordinates X and Y) of the hyperlinkeditems comprised in the hypertext document that is displayed by theopto-touch foil over the page of the physical document.

In our example, the coordinates measured by the opto-touch foil (1004)are around (close to) X=5 and Y=60. They corresponds in the PageHyperlink Table to the hyperlink named “Museums”.

The user workstation Web activates the hyperlink (destination address,URL, . . . ) associated in the Page Hyperlink Table, with the identifiedselected item.

Once the position of the point pressed by the user is identified, thesystem searches in the Page Hyperlink Table what is the closesthyperlinked item coordinates. In our example, the hyperlink labeled“Museums” corresponding to the coordinates X=5, Y=60 is activated. Thus,in this example, a simple pressure near the underlined item “Museums”will automatically activate the following hyperlink on the Web:http://www.worldcities.com/Paris/Museums.html

From this point, the steps (405), (406), (407), (408), (409) and (410)of the method can recursively be used to access, retrieve and displayinformation hyperlinked to this page of the physical document.

In our example, FIG. 11 shows the information that is displayed by theopto-touch foil (1105) when the user activates the hyperlink associatedwith the word “Museums” displayed on the same opto-touch foil. Thisinformation is accessed through the network from the URL:http://www.worldcities.com/Paris/Museums.html and is retrieved by theuser workstation. The displayed information comprises a numbered list ofmuseums of the city of Paris (1103). The locations (1104) of the museumscorresponding to the numerals in this list, are illuminated by theopto-touch foil over the (physical) map of Paris. Each illuminatednumerals is underlined (e.g., 18) for indicating that this numeral is anhyperlink to another hypertext document. In the present example, eachnumeral on the map is related to a museum (1106) in the list (1103) andis hyperlinked to additional information concerning this museum.

To select on the physical page (1102) hypertext documents and to displaythese hypertext documents on the same physical page, a new PageHyperlink Table is build from the received hypertext document on theuser workstation. In the example shown in FIG. 11, the Page HyperlinkTable corresponding to the displayed hypertext document would be likethis:

Doc: 0387 Title: “Interactive Author: David Nash Ford, Editor PAGE: 25Maps of World Cities” Binfield, Berkshire, UK (X, Y) Link: URL: 5 150“Cafeterias” http://www.worldcities.com/ Paris/Cafeterias.html 5 140“Cinemas” http://www.worldcities.com/ Paris/Cinemas.html 5 130“Churches” http://www.worldcities.com/ Paris/Churches.html 5 120“Dancing” http://www.worldcities.com/ Paris/Dancing.html 5 110“Embassies” http://www.worldcities.com/ Paris/Embassies.html 5 100“Hospitals” http://www.worldcities.com/ Paris/Hospitals.html 5 90“Hotels” http://www.worldcities.com/ Paris/Hotels.html 5 80 “Metro”http://www.worldcities.com/ Paris/Metro.html 5 70 “Monuments”http://www.worldcities.com/ Paris/Monuments.html 5 60 “Museums”http://www.worldcities.com/ Paris/Museums.html 5 50 “Pharmacies”http://www.worldcities.com/ Paris/Pharmacies.html 5 40 “Police”http://www.worldcities.com/ Paris/Police.html 5 30 “Pubs”http://www.worldcities.com/ Paris/Pubs.html 5 20 “Post Offices”http://www.worldcities.com/ Paris/Post-Offices.html 5 10 “Restaurants”http://www.worldcities.com/ Paris/Restaurants.html 131 114 “1. Musée duLouvre” http://www.worldcities.com/ Paris/Museums/001.html 145 96 “2.Musée des Arts http://www.worldcities.com/ Décoratifs/MuséeParis/Museums/002.html des Arts de la Mode et du Textile” 162 110 “3.Musée de http://www.worldcities.com/ l'Orangerie” Paris/Museums/003.html153 127 “4. Galerie Nationale http://www.worldcities.com/ du Jeu dePaume” Paris/Museums/004.html 149 107 “5. Musées Nationalhttp://www.worldcities.com/ desTechniques et Paris/Museums/005.html desArts et Métiers” 136 87 “6. Musée de la http://www.worldcities.com/Chasse et la Nature” Paris/Museums/006.html 121 81 “7. Musée Nationalhttp://www.worldcities.com/ Picasso” Paris/Museums/007.html 146 73 “8.Musée de la http://www.worldcities.com/ Serrurerie/MuséeParis/Museums/008.html Bricard” 158 99 “9. Muséehttp://www.worldcities.com/ Carnavalet” Paris/Museums/009.html 141 88“10. Musée national http://www.worldcities.com/ d'Art moderne-CentreParis/Museums/010.html Georges Pompidou” 110 72 “11. Maison de Victorhttp://www.worldcities.com/ Hugo” Paris/Museums/011.html 95 118 “12.Muséum http://www.worldcities.com/ National d'HistoireParis/Museums/012.html Naturelle” 119 131 “13. L'Institut duhttp://www.worldcities.com/ Monde Arabe” Paris/Museums/013.html 125 149“14. Musée National http://www.worldcities.com/ du Moyen-Age”Paris/Museums/014.html 77  62 “15. Musée de lahttp://www.worldcities.com/ Monnaie” Paris/Museums/015.html 149 55 “16.Musée d'Orsay” http://www.worldcities.com/ Paris/Museums/016.html 152 44“17. Musée Auguste http://www.worldcities.com/ Rodin”Paris/Museums/017.html 189 93 “18. Palais de lahttp://www.worldcities.com/ Découverte” Paris/Museums/018.html 193 117“19. Galeries http://www.worldcities.com/ Nationales du GrandParis/Museums/019.html Palais”

FIG. 12, shows how the user (who is interested in receiving moreinformation about the “Musée de la Chasse et la Nature” ), selects thehyperlinked item (1204) (e.g.: the number “6”) by pressing (e.g., bymeans of his fingertip) the opto-touch foil (1203) over it.

By pressing on this point, the opto-touch foil (1203) measures theposition on the page of the point that has been pressed and sends asignal to the user workstation (103) to identify the selected item. Thissignal indicates the position on the page (1202) of the point that hasbeen pressed (near the underlined number: “6”) by the user on theopto-touch foil. In our example, the coordinates that are measured atthis point are around X=136 and Y=87.

The hyperlinked item selected on the opto-touch foil (1203) by the useris identified thanks to the Page Hyperlink Table. In our example, sincethe coordinates measured by the opto-touch foil are around (close to)X=136 and Y=87, they corresponds in the Page Hyperlink Table to thehyperlink named: “6. Musée de la Chasse et la Nature”.

The user workstation Web activates the hyperlink (destination address,URL, . . . ) associated in the Page Hyperlink Table with the selectedhyperlink (i.e., “6. Musée de la Chasse et la Nature”). In our example,the following hyperlink on the Web is activated:http://www.worldcities.com/Paris/Museums/006.html

FIG. 13 shows the information that is accessed through the network fromthe URL: http://www.worldcities.com/Paris/Museums/006.html, retrieved bythe user workstation and displayed by the opto-touch foil (1303). Thehypertext document that is shown comprises the specific information(1303) requested by the user about the: “Musée de la Chasse et laNature”.

From the same physical page (1302), the user can choose to requestinformation for another museum of Paris. Alternatively, he can requestinformation for another different type of service (e.g., “Cafeterias”).Lastly, the user can choose to go to another page of the same physicaldocument.

ALTERNATIVE APPLICATIONS

Other several possible applications of the present invention aredescribed below. Each of these applications basically use the samepreviously described method and system.

Installation and Engineering Instructions

A customer receives a complex computer equipment, with a hard-copyinstallation manual comprising drawings and schemes of the parts andsubassemblies. With the opto-touch foil over any one of these schemes,the user can immediately see certain parts of the complex schemeilluminated with names, legends and instructions. Some of theseilluminated items are identified as hyperlinks items and can be used toaccess additional information on a remote Web server or on a localcomputer. When the customer touches one of those hyperlinked items,instructions showing how the part needs to be installed or serviced, areinstantly displayed over a reserved blank area on the hard-copy manual.A single printed copy with a general view of the equipment is sufficientto navigate with the system according to the present invention. Thecustomer has just to press with his finger on the desired items on thesurface of the installation manual.

Newspapers and Magazines

A subscriber reading a newspaper or magazine, may be interested inseeing additional information associated with the articles he reads.While reading the sports pages (e.g., on the New York Times), key eventscan be instantly recalled and displayed on demand over the samenewspaper (e.g., the time of TV transmission of the next tennis match onWimbledon, etc . . . ) simply by touching an illuminated icon, word,letter, symbol . . . on the article he is reading.

Paper-Based Advertising Linked to On-Demand Multimedia Ads

Many free-toll calls originate from people reading advertisements innewspapers or magazines or in direct mail ads. According to the presentinvention, people can instantly access and visualize product prizes,product details, phone numbers, promotions, discounts and couponsinformation, etc., simply by touching ads printed on paper catalogs.

Apart from the herein described applications, there are many otherpossible applications of this invention.

Phone numbers or phone icons can be displayed and highlighted onnewspaper advertisements, magazines, books and pamphlets, allowingcustomers to make toll-free phone calls simply by touching an itemdisplayed an highlighted on an opto-touch foil.

Home shopping retailers can provide catalogs of specialties. They canhighlight with illuminated icons special offerings and promotions andshow the related information on the same catalogs.

Distance learning applications can access and display additionalinformation or references to complement the illustrations of student'sbooks, etc. . .

What has been described is merely illustrative of the application of theprinciples of the present invention. Other arrangements and methods canbe implemented by those skilled in the art without departing from thespirit and scope of the present invention.

1. A method of selecting electronic information from a physicaldocument, and for displaying said selected electronic information overthis physical document, said method comprising: identifying a physicaldocument; identifying a page of said physical document; identifying andlocating an electronic document associated with the identified page byreferring to a document hyperlink table; said document hyperlink tablecomprising for each page of the physical document the identification andlocation of an electronic document; accessing and retrieving theelectronic document associated with the identified page; and displayingthe retrieved electronic document by means of an opto-touch foil placedover the identified page in conjunction with providing at least partialcontent of the identified page for viewing via the opto-touch foil. 2.The method according to claim 1, wherein the electronic documentassociated with the identified page comprises one or more hyperlinkeditems, said method further comprising: creating a page hyperlink tablefor the identified page; identifying the one or more hyperlinked itemsin the electronic document associated with the identified page; and foreach identified hyperlinked item: identifying and locating informationassociated with said identified hyperlinked item; determining theposition on the opto-touch foil of said identified hyperlinked item; andstoring the identification and location of the information associatedwith said identified hyperlinked item and the position of saididentified hyperlinked item on the opto-touch foil in the page hyperlinktable.
 3. The method according to claim 2, further comprising: aligningsaid opto-touch foil over said identified page of said physicaldocument; determining the position of a point pressed on said opto-touchfoil; identifying a selected hyperlinked item that corresponds to theposition of the point pressed on said opto-touch foil by referring tothe page hyperlink table; identifying and locating informationassociated with the selected hyperlinked item by referring to the pagehyperlink table; accessing and retrieving the information associatedwith the selected hyperlinked item; and displaying the retrievedinformation by means of the opto-touch foil aligned over the identifiedpage.
 4. The method according to claim 3, wherein the method is used ina user system that is connected to a communication network comprisingone or more servers, and wherein the information associated with thehyperlinked items is located on said one or more servers or locally onthe user system.
 5. The method according to claim 4, wherein theinformation associated with the hyperlinked items is located on the usersystem.
 6. The method according to claim 5, wherein identifying andlocating information associated with the selected hyperlinked item byreferring to said page hyperlink table comprises: determining adestination address in the communication network where the informationassociated with the selected hyperlinked item can be accessed byreferring to said page hyperlink table.
 7. The method according to claim6, wherein: said communication network is an Internet Protocol (IP)network; said servers are Web servers; said user system comprises a Webbrowser; said destination address is an Uniform Resource Locator (URLaddress); and said information is a Web page.
 8. The method according toclaim 1, wherein the physical document is a hard-copy document or anytype of physical surface such as an engraved, printed, painted, writtensurface, or any material such as paper, wood, plastic and of any formsuch as a newspaper, magazine, book, catalog, geographical map,photograph, painting.
 9. The method according to claim 1, wherein thedocument hyperlink table associated with the physical document comprisesadditional information related to said document such as title, author,date and wherein the method further comprises: accessing said additionalinformation.
 10. A system comprising: an opto-touch foil; a user systemcomprising: means for selecting a page of the physical document; meansfor retrieving an electronic document associated with the selected page;and means for displaying the retrieved electronic document using theopto-touch foil placed over or under the selected page in conjunctionwith providing at least partial content of the selected page forviewing; and a connection between said opto-touch foil and said usersystem.
 11. The system according to claim 10, wherein the opto-touchfoil is sensitive to the pressure exercised over any point of itssurface and comprises a transparent film with light emitting elementsfor displaying electronic information.
 12. A method for displayingselected electronic information on a physical document, the methodcomprising: identifying a physical document; selecting a page of thephysical document; retrieving an electronic document associated with theselected page; and displaying the retrieved electronic document using anopto-touch foil placed over or under the selected page in conjunctionwith providing at least partial content of the selected page forviewing.
 13. The method according to claim 12, wherein the retrievedelectronic document is located through use of a lookup table.
 14. Themethod according to claim 13, wherein the lookup table comprises adocument hyperlink table that associates respective ones of a pluralityof electronic documents with each page of a physical document.
 15. Themethod according to claim 14, the method further comprising: identifyinga hyperlinked item in the electronic document associated with theselected page; locating information associated with the identifiedhyperlinked item; determining the position on the opto-touch foil of theidentified hyperlinked item; and storing both the location of theinformation associated with the identified hyperlinked item and theposition of the identified hyperlinked item.
 16. The method according toclaim 15, further comprising: placing the opto-touch foil over or underthe identified page of the physical document; identifying the positionof a point pressed on the opto-touch foil; identifying a selectedhyperlinked item that corresponds to the identified position byreferring to the stored position information; locating informationassociated with the selected hyperlinked item by referring to the storedlocation information; retrieving the information associated with theselected hyperlinked item; and displaying the retrieved information bymeans of the opto-touch foil.